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Student Engagement with a Science Simulation: 
Aspects that Matter 


Susan Rodrigues*' and Eugene Gvozdenko 2 

It is argued that multimedia technology affords an opportunity to bet¬ 
ter visualise complex relationships often seen in chemistry. This paper 
describes the influence of chemistry simulation design facets on user 
progress through a simulation. Three versions of an acid-base titration 
simulation were randomly allocated to 36 volunteers to examine their in¬ 
teractions with the simulation. The impact of design alterations on the 
total number of interactions and their patterns was analysed for the fol¬ 
lowing factors: (a) the place of a feature on the screen, (b) alignment of 
the sequence of instructions, (c) additional instruction before the simula¬ 
tion, (d) interactivity of a feature. Additionally, interactions between in¬ 
dividual factors, such as age, prior experience with science simulations 
and computer games, perception of the difficulty of science simulations, 
and general subject knowledge, on one hand, and the efficiency of using 
the simulation, on the other hand, were examined. The findings suggest 
that: (a) centrality of the position of an element significantly affects the 
number of interactions with the element, (b) re-arranging the sequence 
of instructions on the screen in left-to-right order improves the following 
of instructions, (c) providing users with additional written advice to fol¬ 
low numbered instructions does not have a significant impact on student 
behaviour, (d) interactivity of a feature was found to have a strong positive 
correlation with the number of interactions with that feature, which war¬ 
rants a caution about unnecessary interactivity that may hinder simula¬ 
tion efficiency. Surprisingly, neither prior knowledge of chemistry nor the 
age of the participants had a significant effect on either the number of 
interactions or the ability to follow on-screen instructions. 
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Interakcija studentov z naravoslovnimi simulacijami: 
pomembni vidiki 


Susan Rodrigues* in Eugene Gvozdenko 


Multimedijska tehnologija naj bi nudila moznosti boljse predstavitve 
kompleksnih odnosov med pojmi, ki se pogosto pojavljajo pri kemiji. 
Prispevek podaja vpliv dizajna kemijske simulacije na napredek posa- 
meznika pri uporabi simulacije. Tri razlicice simulacije na temo titracije 
kisline z bazo so bile nakljucno predstavljene 36 prostovoljcem, da bi 
raziskali njihovo interakcijo s simulacijo. Vpliv treh razlicnih oblik di¬ 
zajna simulacij na skupno stevilo in vzorec interakcij posameznika s 
simulacijo je bil analiziran glede na: a) mesto elementa na zaslonu, b) 
polozaj zaporedja navodil, c) dodatna navodila pred simulacijo in d) 
interaktivnost elementa. Dodatno so bile raziskane se povezave med 
starostjo, predhodnimi izkusnjami z naravoslovnimi simulacijami in 
racunalniskimi igrami, dojemanjem zahtevnosti naravoslovnih simula¬ 
cij, znanjem kemijskih pojmov in ucinkovitostjo studentov pri uporabi 
simulacij. Ugotovitve kazejo, da: a) centralna postavitev dolocenega ele¬ 
menta v simulaciji pomembno vpliva na stevilo interakcij s tern elemen- 
tom, b) razporeditev zaporedja navodil na zaslonu od leve proti desni 
izboljsa sledenje navodilom, c) dodatna pisna navodila uporabnikom, 
da naj sledijo ostevilcenim navodilom, ni imela pomembnega ucinka 
na vedenje studentov, d) korelacija med interaktivnostjo elementa in 
stevilom interakcij s tern elementom je pozitivna, mocna in pomembna, 
kar kaze na to, da je treba biti pri snovanju simulacij previden, da ne 
omogocamo nepotrebnih interaktivnosti, ki lahko zavirajo ucinkovitost 
simulacije. Presenetljivo je, da predznanje kemije in starost udelezencev 
nista imela pomembnega vpliva na stevilo interakcij in zmoznost 
sledenja navodilom na zaslonu. 

Kljucne besede: ucenje, kemija, izobrazevalne simulacije, oblikovanje 
simulacij, interaktivnost, navodila 
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Introduction 

Information communication technology (ICT) has become ubiquitous 
as it has become more affordable and more powerful (Madden et al., 2005). By 
2008, approximately 66% of British homes had Internet connection, (Office for 
National Statistics, 2008) and in more recent years, a change in connection to 
the Internet in the form of broadband has reduced the need for homes to have 
a computer-dedicated line, increased the speed of data transfer, and allowed 
for increased use of multimedia within web pages. Valentine, Marsh and Pattie 
(2005) found that the majority of children used their home computer for school 
work. Over ten years ago, when Rideout, Foehr and Roberts, (1999) asked a rep¬ 
resentative sample of American children aged 8-r8 which medium they would 
take to a desert island, the preferred choice was a computer with Internet ac¬ 
cess. Thus it is not surprising that over recent decades, schools, researchers and 
policy makers have all shown growing interest in the use of ICT to support 
classroom teaching and learning. 

As a consequence, we have seen increasing literature reporting on vari¬ 
ous forms of ICT for science education. This literature has included reporting 
on the use of audience response systems (Rodrigues, Taylor, Cameron, Syme- 
Smith, & Fortuna, 2010), dataloggers (Tortosa, Pinto, & Saez, 2008), email (Van 
derMeij & Boersma, 2002), the Internet (Mackenzie, 20ro), modelling (Pallant 
& Tinker, 2004), simulations (Eilks, Witteck, & Pietzner, 20ro), virtual charac¬ 
ter research (Rebolledo-Mendez, Burden, & de Freitas, 2008) and whiteboards 
(Redman, McDougal, & Rodrigues, 2oro). Within this body of work, one can 
also find research linking the culture of informal computer games, student in¬ 
terest and the development and design of appropriate ICT for chemistry (see 
Prensky, 2004; Grimley et al., 2010), as well as work on attitudes (Tondeur, Van 
Keer, van Braak, & Valcke, 2008). In the present paper, we consider more than 
just the motivational aspect; we look at the process of engagement and the in¬ 
fluence of the design element in terms of supporting cognitive and skill devel¬ 
opment in science education. 

Designers’ views of learners and their assumptions about learning theo¬ 
ries, learning processes and learning practices ensure that content and pedago¬ 
gy are intertwined before the technology reaches the classroom (Segall, 2004). 
Consequently, multimedia design for school purposes has been explored and 
continues to be explored, resulting in a debate about the influence of various 
factors in supporting or hindering learning. 

Mayer, Sobko and Mautone (2003) define multimedia learning as the 
use of at least two different types of media (graphics, audio, video and text) in 
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presenting information. Clarke and Mayer (2003), Ginns (2005) and Moreno 
(2006) reported a modality principal and suggested that graphical informa¬ 
tion explained by onscreen text and audio narration led to cognitive overload 
and was therefore detrimental to learning. In more recent times, studies (see 
Dunsworth & Atkinson, 2007; Sanchez & Garcia-Rodicio, 2008) suggest that 
there is no difference in performance based on the presence or absence of au¬ 
dio narration. Eilks et ah, (2010) suggested that technology that allows for a 
seamless interchange between tables, charts, graphs and model displays could 
support conceptual linking between these representations. Ploetzner, Bodemer 
and Neudert (2008) suggest that the required high transfer rate may, unfortu¬ 
nately, result in a limited attention span. Testa, Monroy and Sassi (2010) suggest 
that graphs depicted in textbooks are ‘cleaned’ of redundant details/irregulari- 
ties, whereas technology-generated real-time graphs include ‘noise’, resulting in 
some learners finding them challenging to interpret. Indeed, the argument per¬ 
taining to computer-based graphing exercises has had a long lifespan. For ex¬ 
ample, the Brasell (1987) study suggested that a delay in display, even if less than 
30 seconds, resulted in subduing nearly all students, for they demonstrated less 
engagement and became preoccupied with procedural issues. However, Beich- 
ner (1990) suggested that student engagement could be lowered if the software 
constructed the graphs. Schnotz and Bannert (2003) suggested that picture use 
in multimedia learning processes may not be beneficial in every case, while 
Schwartz, Andersen, Hong, Howard and McGee (2004) and Azevedo (2004) 
suggest the use of non-linear learning environments may result in inadequate 
metacognitive competencies. Paivio’s dual coding theory (2006) suggests that 
multiple references to information with connections between verbal and non¬ 
verbal (imagery) processing improves the learning process. Chandler and 
Sweller’s (1991, 1992) ‘split attention’ effect (with the learner addressing multi¬ 
ple information sources before trying to integrate the segments to make them 
intelligible) and their ‘redundancy’ concept suggest that disparate sources may 
generate cognitive overload. Paivio (2006), Chandler and Sweller (1991, 1992) 
may appear to hold contradictory views, but both sets of ideas seem feasible 
and at present neither explanation has more currency than the other. 

In light of these various arguments, and given the growing use and 
production of simulations and animations in school chemistry, we decided to 
explore the influence of chemistry simulation design facets on user progress 
through a simulation. It is argued that multimedia technology affords an oppor¬ 
tunity to better visualise complex relationships. We were interested in the scope 
of this opportunity and hence developed the following research questions: 

• What are the differences in the nature of student interactions associated 
with an altered simulation design format? 
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• What are the effects of the changes in instruction formats on the process of 
students’ engagement behaviour? 

• How effective are additional written instructions before the simulation? 

• How does altering the position of controls on the simulation screen affect 
students’ engagement with the simulation? 

Method 

Participants 

The convenience sample included 57 volunteers from four schools and 
one tertiary institution. The data collected did not identify the volunteers on a 
personal level. They were anonymously allocated individual codes when they 
accessed the website and the different institutions were recognised by the log. 
The volunteers were asked to provide their age, gender, science subject (science, 
chemistry, physics, biology) and class/tertiary level, as well as to indicate their 
previous ICT experience and complete five multiple choice chemistry questions 
pre-simulation use and post-simulation use. Fifty-seven volunteers submitted 
required information and 36 of them interacted with the simulation. The data 
collected from the volunteers who submitted questionnaires and actually in¬ 
teracted with a simulation provided were used for the analysis presented in the 
present paper. 

Among the 36 participants, there were 19 students aged 13-15 years (sec¬ 
ond year of secondary school) and 15 students aged 16 and over. Two partici¬ 
pants did not indicate their age. There were roughly equal numbers of male 
and female participants (17 females and 16 males) using this simulation. Three 
participants did not supply details about gender. 

Table 1: Descriptive statistics of the sample. 


Sample description 


Simulation versions 

1 

2 

3 


Male 

5 

5 

6 

Gender 

Female 

1 

8 

8 


Not indicated 

0 

1 

2 


13-15 

2 

7 

10 

Age 

16 and over 

3 

7 

5 


Not indicated 

1 

0 

1 
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Chemistry 

0 

4 

5 

Science 

Physics or Biology 

1 

2 

0 


Combination 

2 

8 

8 


Not indicated or none 

2 

0 

3 


Yes 

4 

8 

11 

Playing PC games 

No 

1 

5 

4 


Not indicated 

1 

1 

1 

Prior experience in 

Yes 

4 

6 

12 

using simulations in 

No 

1 

7 

3 

Science lessons 

Not indicated 

1 

1 

1 


Research design 

Professor Thomas Greenbowe (2005) kindly provided access to the 
code for two of his flash-based simulations (a titration and reactivity of met¬ 
als) available on the internet as learning resources aimed at introducing college 
chemistry (general chemistry). We modified the code to create three versions 
of each simulation and to add a facility for monitoring users’ interactions with 
the simulations. A system was created that randomly allocated one version of 
the two simulations to each user as they accessed the website. A log of all mouse 
clicks and interactions with the simulation controls (buttons, sliders, text fields 
and selection boxes) was generated for each user. The computer tracked the 
time that the user spent on each stage and on each particular element of the 
simulations. 

This behind-the-scenes recording of activity was chosen for three rea¬ 
sons. Firstly, we felt it would be less intrusive, and that it therefore had the 
potential to generate more reliable data. Secondly, collecting images of school 
children is increasingly discouraged by local authorities. Thirdly, the url was 
available for use outside the classroom, and filming its use in that milieu would 
be impractical. 

Each user had to complete a pre-simulation questionnaire (specific to 
the chemistry topic for the simulation being viewed) before being randomly 
allocated one of three versions of the simulation. After the simulation, they 
were asked to complete a post-simulation test and a post-questionnaire. The 
pre-simulation and post-simulation chemistry questions were based on those 
found in standard textbooks. However, these questions are not discussed here, 
as the present paper focuses on patterns of interaction and engagement. Figure 
1 provides an overview of the sequence. 







C-E-P-S Journal I Vol.l I N“4| Year 2011 


Methodology 

Three simulation versions_ 

| Pretest questionnaire | 

[ Pie simulation test 1 



Posttest questionnaire 


Figure 1: Experimental design. 

The simulations 

The simulations we used probably best fit within the Thomas and 
Hooper (1991) category of experiencing simulations’. Experiencing simulations 
model particular scenarios, allowing students to manipulate factors to see their 
impact or influence. The simulations we used were representative of many com¬ 
mon types of simulations used in school science lessons. However, by select¬ 
ing an acid-base titration simulation aimed at ‘college level’ we were able to 
explore the influence of age and, consequently, prior experience factors on user 
ability to follow instructions, as while the acid-base titration would be familiar 
to older students it would be completely novel to the younger students in our 
sample cohort. The y-year-old students would have encountered the terms acid 
and base, but in our experience they would not have conducted a titration dur¬ 
ing practical or wet-lab work in schools. Our sample also included first-year 
university chemistry undergraduate students, who almost certainly would have 
conducted titrations during their senior years at school and during their first 
year at university. 

The acid-base titration simulation had three versions: the original ver¬ 
sion (Version r), a modified version (Version 2) that included a one paragraph 
pre-text advising students to pay attention to particular aspects (as can be seen 
in Figure 2), and another modified version (Version 3) that had altered posi¬ 
tions for specific elements on the screen (as can be seen in Figure 3). 
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The following is the excerpt paragraph that appeared on the webpage 
before the Version 2 titration simulation loaded: 

“When you click on the button below you will see a simulation that rep¬ 
resents a titration. To make the simulation work you must follow the 
numbered instructions in sequence. So start with instruction 1, then 2, 
then 3, etc. Some instructions have tabs. You must place the mouse on 
the tab and drag it open”. 

In Version 3, a menu tab, also identified with the number 3 on the simu¬ 
lation “Select the Acid and Base”, was converted from a ‘pull out tab’ menu to 
a fixed position, visible menu. The position of other items on the screen was 
also modified so that the sequence of instructions was aligned with a common 
reading pattern (horizontal sequence of left to right) (Gvozdenko et al., 2010). 


Determination of the Molarity of 
an Acid or Base Solution 


1. Select Type 
of Reaction 


O Strong Acid vs. Strong Base 
O Weak Acid vs. Strong Base 


2. Fill the 
Burette with 

O Acid 
O Base 


Total 

Volume of 


6. Calculate and 
Enter Molarity of 


5. Push 
Slider Up 
to Add a 
Volume of 


Jml 




Figure 2: Titration simulation Versions 1 and 2. 
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Determination of the Molarity of 
an Acid or Base Solution 


50 


Molarity of Volume of 


1. Select Type 
of Reaction 

O strong Acid vs. Strong Base 
O Weak Acid vs. Strong Base 


4. Select the 
Indicator 

O Methyl Orange 
OPhenolphthalein 


5. Push 
Slider Up 
to Add a 
Volume of 



3. Select the Acid 
and Base 


Acid 

Base 

O 

HCI 

O 

KOH 

O 

h 2 so 4 

O 

NaOH 

O 

HN03 

O 

Sr(OH) 2 

O 

HCI04 

O 

Ba(OH) 2 


2. Fill the 
Burette with 

O Acid 
O Base 



Figure 3: Titration simulation Version 3. 


Data analysis 


We used one-way unrelated analysis of variance (ANOVA) to determine 
whether the different versions of the acid-base titration simulation had an impact 
on how students followed the sequence of instructions. Each simulation version 
involved a separate and unrelated sample of subjects, so we assumed equal popu¬ 
lation variation and normal distribution of our random population within the 
different version cohort. The one-way ANOVA allowed us to deduce the mean 
for the three versions and then compare these means between the versions. Cal¬ 
culating the one-way ANOVA and the variation between scores meant we could 
compare the variation between sample means for each simulation version. In the 
null hypothesis, the assumption was that the mean for Version 1 was the same as 
the mean for Version 2 and Version 3. However, if the one-way ANOVA showed 
that the variation between the samples was bigger than the variation in the popu¬ 
lation, we would have to accept the alternative hypothesis, i.e., that the variation 
was due to an independent variable. If the variability was statistically significant, 
the findings would indicate that the independent variable was having an effect. 
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We used SPSS to separate the groups for analysis, creating a grouping 
variable called simulation, and represented each of the three Versions as 1, 2 or 
3. As would be expected, the time required to complete each respective simu¬ 
lation version was entered under a variable named ‘Time’. Means and stand¬ 
ard deviations were determined for each version, and by using Levene’s Test of 
Homogeneity of Variance we verified that the assumption of homogeneity of 
variance was met. 

A modified grounded theory approach (see Strauss and Corbin, 1998) al¬ 
lowed us to group track patterns as they emerged from the logged data: a preliminary 
reading of the tracks allowed for familiarisation of the whole data set of 36 tracks. At 
this stage, we suggested explanations, which was followed by a closer reading of the 
tracks that led to interpreting and coding into themes. To ensure rigour, the data 
analysis was triangulated. As two independent researchers, we reviewed the data and 
then reflected on and compared the themes that emerged from our independent 
analysis. This process helped us to develop perspectives while reducing subjectivity 
bias. Themes that emerged from the tracks as common or typical, resulting in what 
van Manen (1990) called ‘control and order’, allowed us to generate what Polking- 
horne (1988) called ‘plotlines’ for the collated tracks. These plotlines inform the writ¬ 
ing presented in this article. When reviewing the tracking data, we were particularly 
interested in the nature of actions and steps taken by the users, as we were interested 
in the nature of engagement with the different versions of the simulation. Tracking 
their engagement could also tell us about the influence of particular design elements, 
thus allowing for an evaluation of effectiveness and performance as gauged by the 
pre-simulation and post-simulation tests. 

Findings 

Our findings are based on the tracks generated by student engagement 
and actions when using a randomly assigned version of the titration simulation. 
As the simulations were allocated randomly to volunteers, six students com¬ 
pleted a pre-survey and engaged with Version 1, while 14 students completed a 
pre-survey and engaged with Version 2 and 16 students completed a pre-survey 
and engaged with Version 3. 

Our findings show that 62% of the participants thought the titration simu¬ 
lation was equally as interesting as a computer game, and 82% believed that sci¬ 
ence simulations were easy. Hence it could be argued that the students involved 
were not novices in using simulations, and perceived themselves to be efficient 
simulation users. Despite this, the tracks showed that, unfortunately, only one 
participant reached the correct response in the field CALC OK at STEP 6. 
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Positioning instructions and icons 

An analysis of the tracks showed that if a button that controls the drop- 
wise addition from the burette is at a more central location it increases the 
number of interactions with that particular control by approximately 25%. The 
analysis also showed that having control elements in a side position decreased 
the number of interactions. 

Analysis showed that converting a tab menu (that slid out) into a fixed 
menu resulted in a decrease in the number of overall interactions, including 
non-productive interactions, by 30-40%. 

The data collected also allows an analysis of the relationship between 
student responses (in terms of gender, age, computer game experience and 
simulation user experience) and two measures of their behaviour and activity 
when using the simulation: 

(a) the pattern of engagement with the simulation inputs/controls, 

(b) the total number of interactions between a student and the simulation. 

A one-way unrelated analysis of variance (ANOVA) found that the sim¬ 
ulation version had a significant effect on how students followed the order of 
the instructions (F2,29=3.69, p< 0.05). The extent to which students followed 
a recommended sequence of controls was significantly higher among the stu¬ 
dents using simulation Version 3 (M=4-24, SD=i. 43), with 16 students, than for 
students using simulation Version 2 (M=2.85, SD=i-46), with r4 students. This 
was independent of age or gender. The extent to which students followed the 
intended sequence of controls was also higher with students using Version 3 in 
comparison with students using Version 1 (M=3.20, SD=i.ro). However, as in¬ 
dicated previously, the Version 3 and Version 1 comparative finding warrants a 
degree of care, as there was a smaller number of students (n=6) using Version 1. 

Contrary to our expectations, prior experience in playing computer 
games had no significant effect on how students followed the order of the in¬ 
structions (Fi,29=0.132, p= 0.719). However, prior experience in playing games 
had a significant effect on the number of interactions (Fi,29=4.8i, p= 0.036), 
with those students who indicated that they did not play computer games 
(n=io, M=40, SD=33) having nearly three times fewer interactions than those 
who indicated that they played computer games (n=2i, M=i29, SD=23). 

The students who had previous experience (n=n, M=68, SD=64) with 
simulations in a lesson were on average engaged in more interactions with the 
simulation than those who did not (n=20, M=n8, SD=i29). This effect was not 
statistically significant. A one-way unrelated analysis of variance (ANOVA) 
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found that prior experience in using simulations in a lesson had a significant ef¬ 
fect on how students followed the order of the instructions (Ft, 29=4.21, p<o. 05). 
Perhaps, as to be expected, the students with no experience in simulation use in 
classrooms (n=io, M=2.82, SD=i. 47) on average followed the order of the con¬ 
trols less efficiently than those with prior experience (n=20, M=3-95, SD=i.54). 

Student perception of easiness’ in using a simulation was found to have 
a significant effect on the number of interactions ^2,24=5.31, p< 0.05). The stu¬ 
dents who thought that it was “very easy” to use a simulation (n=2, M=336, 
SD=202) on average had twice as many interactions than those who thought 
that simulations are “easy” (n=20, M=90, SD= 97) or “not easy” (n=5, M=ioo, 
SD=8 5 ). 

The analysis of data showed that age did not have any significant effect 
on student behaviour patterns (F3,27=0.274,^=0.843). 

This would suggest that regardless of whether or not the students had 
previously encountered the chemistry (acid-base titrations) there was no sig¬ 
nificant effect on behaviour, which would imply prior knowledge of chemistry 
did not have a significant effect on either the number of interactions or the 
order in following instructions. 

Patterns of behaviour 

Two of the students using Version 1 and two of the students using Ver¬ 
sion 2 did not appear to pay attention to the ‘number sequence’ associated with 
the instructions. These numbered instructions were intended to steer them and 
guide the decisions they made with respect to their process order. What was no¬ 
ticeable was that the proportion of those wrongly following numbered instruc¬ 
tions was less for the cohort using Version 2 (simulation with pre-direction) 
than the cohort using Version 1, but not less than the cohort using Version 3. 
The difference in behaviour between the three versions showed that 10 of the 
14 participants using Version 2, which is over two thirds, had chaotic behav¬ 
iour patterns. In contrast, only three of the 16 participants using Version 3 had 
chaotic behaviour patterns, while 11 of those using simulation Version 3 (fixed 
position openly displayed menu and modified reading pattern) followed the 
steps sequentially. 

Interestingly, despite having directions to steer them towards the pro¬ 
cess sequence order, only one of the participants managed to follow the steps, 
and 10 of the 14 students who used Version 2 either showed chaotic behaviour 
or only managed to complete step/instruction 2 in sequence. In addition, three 
of the participants using Version 2 (which provided pre-direction before they 
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commenced using the simulation) took between 2.5 and 3 minutes to find and 
operate the sliding tab menu (instruction 3). There appeared to be a similar age 
distribution across Version 2 and Version 3, so the chaotic patterns were not 
due solely to age and possible prior experience. In fact, there were five first-year 
undergraduates using Version 2, and only one of them reached step 4 in simula¬ 
tion Version 2. 

Conclusion 

Our findings involve a small sample size, and with this come the usual 
caveats regarding drawing generalisations. Nevertheless, our findings suggest 
that simply providing instructions for students to read prior to using a simula¬ 
tion does not necessarily result in the students following the sequence in the 
simulation as designed. However, if the design is less ambiguous, for example, a 
‘pull out tab’ menu when converted to a fixed position visible menu, the result 
is better engagement. It seems that additional instructions before a simulation 
cannot compensate for ambiguity in simulation design: despite being given di¬ 
rections advising them of the process sequence order, most users of the original 
versions (1 and 2) showed chaotic engagement behaviour tracks. In contrast, 
the modified Version 3, with a left-to-right and top-to-bottom aligned sequence 
of menu controls and a fixed visible menu, saw only one fifth of the cohort 
displaying chaotic behaviour, with most users following the intended sequence. 

The presence of an interactive component in a simulation needs to be 
justified by a learning goal. While visual demonstration involving chemical lab¬ 
oratory tools, such as a probe or a thermometer, or a depiction of atom move¬ 
ment in different chemical solutions could aid learning, the interactive sliding 
out menu tab was a hurdle for some students who clicked multiple times on the 
tab control. 

Our findings also show that age did not have any significant effect 
on the student behaviour patterns. Given that some of the participants were 
undergraduate degree-level students, this would suggest that regardless of 
whether or not they had previously encountered acid-base titrations there was 
no significant effect on engagement behaviour. This implies that prior knowl¬ 
edge of chemistry did not have a significant effect on either the number of 
interactions or the order in following instructions. These findings suggest that 
simulation design is therefore crucial if, for example, a simulation is to be used 
for assessment purposes. For a student may have the requisite subject content 
knowledge to enable them to undertake a wet-lab practical, but when they 
encounter a simulated version of that wet-lab practical it may be their ability 
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to engage effectively with the technology that hinders their ability to perform 
to capacity. 

Furthermore, given their apparent self-perception of their computer 
skills, they may underestimate the impact of the technological skill required 
to use a chemistry simulation, if they fail to pay attention to the instructions. 
Hence, while we would advocate that when creating simulations designers need 
to take care to ensure that what appears obvious to them is equally obvious to 
the user, we would also suggest that users, in this case students, need to start 
taking responsibility and understand that, while their chemistry may be sound, 
it may be their inability to follow instructions that affects their assessment if the 
assessment involves a simulation. 
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